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Abstract 

Quartz  and  rubidium  oscillators  disciplined  by  the  signals  of  the  Global  Positioning  System 
satellites  (GPSDOs)  are  increasingly  used  as  reference  standards  in  calibration  centers  and  in 
telecommunication  networks,  thanks  to  their  cesium-like  long-term  instability,  and  several 
investigations  are  undergoing  in  the  metrological  laboratories  of  different  countries  concerning 
their  use  for  the  traceability  to  the  national  standards  of  time.  Some  of  these  devices  were  tested 
in  the  past  at  the  Time  and  Frequency  Laboratory  of  IEN,  as  regards  to  their  use  as  frequency 
and  time  references  in  secondary  laboratories  and  their  traceability  to  the  Italian  standard  of 
time. 

In  further  investigations  performed,  evidence  was  found  of  short-term  frequency  instabilities, 
not  previously  detected,  mainly  due  to  temperature  effects  and  to  the  disciplining  algorithms 
used,  that  must  be  taken  into  account  especially  in  the  frequency  calibration  field.  The  long  and 
short-term  instability  results  obtained  at  IEN  on  some  GPSDOs,  that  show  the  real  uncertainty 
limits  in  calibration,  are  reported  in  this  paper.  They  are  also  checked  by  means  of  a  field-trial 
on  frequency,  carried  on  among  some  Italian  Calibration  centers  equipped  with  GPSDOs  or 
other  frequency  references,  using  either  a  free  rubidium  or  a  quartz  oscillator  as  travelling 
standards. 


INTRODUCTION 

The  traceability  in  Italy  to  the  national  time  standard  UTC(IEN),  realized  by  IEN,  of  the  secondary 
standards  maintained  in  the  calibration  centers  accredited  by  the  Italian  Calibration  Service  (SIT),  can  be 
obtained  by  means  of  different  synchronization  systems,  one  of  the  most  used  now  being  the  GPS  in  the 
common-view  technique  or  as  a  disciplining  medium  to  stabilize  the  frequency  of  high  quality  oscillators 
[11- 

In  these  disciplined  oscillators,  the  frequency  offset  and  the  drift  are  continuously  compensated  and 
therefore  a  specific  approach  for  the  traceability  issue  has  to  be  followed  especially  if,  as  in  the  case 
under  study,  the  time  signals  used  in  the  disciplining  process  are  not  originated  by  the  national  standard. 
The  problem  of  establishing  the  traceability  of  the  GPSDOs  was  faced  in  the  past  at  IEN  performing 
several  tests  on  devices  of  different  manufacturers,  leading  to  a  definition  of  their  accuracy  and  stability 
limits  and  of  a  measurement  protocol  for  their  on  site  calibration  [2], 

In  1998,  the  issue  of  the  frequency  accuracy  and  stability  of  GPSDOs  as  stand-alone  frequency  standards, 
over  the  observation  times  involved  in  the  calibration  process,  has  been  examined  at  IEN  through  an 
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extensive  investigation  on  eight  GPSDOs  of  different  manufacturers,  equipped  either  with  a  quartz  or  a 
rubidium  oscillator.  The  results  of  these  studies  and  the  consequences  on  the  uncertainty  budget  of  the 
centers  accredited  for  frequency  are  reported  in  the  following,  together  with  the  evaluation  of  the  long¬ 
term  behavior  of  some  of  these  devices  operating  in  Italian  laboratories,  as  obtained  by  implementing  the 
daily  measurement  protocol  agreed  upon.  Future  work  envisaged  in  this  field  is  also  outlined. 

LONG-TERM  CHARACTERIZATION  OF  GPSDOs 

The  frequency  and  time  interval  measurements  performed  to  estimate  the  GPSDOs  specifications  have 
been  referred  to  UTC(IEN)}and  the  differences  between  UTC(IEN)  and  the  GPS  time  scale  have  been 
determined  with  the  NBS/GPS  receiver  used  for  the  international  traceability  and  performing  the  BIPM 
common-view  (CV)  tracking  schedule  for  Europe.  The  mean  frequency  deviation  between  the  IEN  and 
the  GPS  time  scales  has  always  been  well  below  MO'13  during  the  instruments  testing  periods. 

The  measurement  results  reported  in  the  following  were  obtained  by  means  of  a  Stanford  SR620  Time 
Interval  and  Frequency  Counter,  supplying  an  external  10  MHz  derived  from  UTC(IEN)  as  a  time  base. 
For  observation  times  up  to  1000  s,  frequency  measurements  were  performed  using  an  additional  phase 
difference  multiplier  to  increase  the  resolution*,  meanwhile  for  longer  observation  times,  time  interval 
measurements  were  used, started  by  a  1PPS  from  UTC(IEN)  time  scale  and  closed  by  a  1PPS  supplied  by 
the  GPSDO  under  test. 

The  indoor  equipment  and  the  measurement  system  was  inside  the  Time  and  Frequency  Laboratory 
where  the  temperature  has  been  maintained  at  (23  ±  1,5)  °C  and  the  AC  power  stabilized  at  (220  ±  5)  V. 
Eight  devices  from  three  different  manufacturers  have  been  analyzed:  two  equipped  with  an  ovenized 
crystal  oscillator  (OCXO)  and  labeled  in  the  following  as  A  and  B,  two  with  a  low  drift  crystal  oscillator 
(BVA)  and  labeled  as  C  and  D,  and  four  with  a  rubidium  frequency  standard  (Rb)  named  E,  F,  G  and  H. 
They  have  been  checked  as  regards  their  capability  to  reproduce  GPS  time,  their  short-  and  long-term 
instability,  their  frequency  accuracy  and  the  supplying  of  information  useful  to  establish  a  traceability  to 
an  external  reference  standard.  In  some  cases  the  devices  have  been  operated  under  their  default 
conditions,  in  others  the  reference  coordinates  of  the  IEN  site  have  been  inserted  and  the  receiver  forced 
to  operate  in  “time  mode.” 

The  results  obtained  from  daily  time  measurements  and  the  statistics  about  the  long-term  frequency 
behavior  of  the  GPSDOs  under  evaluation  at  the  IEN  laboratory  are  reported  in  Table  1,  which  clearly 
shows  the  effect  of  the  GPS  disciplining  process,  that  has  compensated  for  the  oscillators  frequency 

offsets  and  drifts.  It  can  be  noticed  in  fact  that  the  mean  relative  frequency  deviations  y  of  the 
GPSDOs,  computed  over  the  whole  period  from  a  set  of  daily  averaged  frequency  deviations,  are  always 
negligible  in  comparison  with  their  uncertainty  sy  estimated  as  the  standard  deviation  of  the  daily 
frequency  values,  and  in  most  cases  approach  GPS. 

Some  of  these  devices  have  been  afterwards  remotely  tested  in  the  calibration  centers  where  they  are 
operating  as  reference  standards.  The  measurement  protocol  for  the  remote  frequency  calibration  of 
GPSDOs  versus  the  national  time  and  frequency  standard,  that  requires  a  calibration  center  to  perform  a 
daily  series  of  24  time  interval  measurements  between  the  local  1PPS  and  the  GPS  time  signals  provided 
by  the  GPSDO,  has  been  already  implemented  in  six  calibration  centers  that  send  the  results  monthly  to 
IEN.  Each  measurement  cycle  starts  at  the  beginning  of  the  hour  and  consists  of  60  consecutive  time 
interval  measurements;  at  IEN,  48  daily  GPS  measurement  -  lasting  13  minutes  each  -  are  performed 
according  to  the  BIPM  CV  schedule  for  Europe.  From  these  two  data  ensembles,  a  mean  daily  time 
difference  between  UTC(IEN)  and  the  1PPS/GPSDO  is  computed  and  the  daily  average  frequency 
deviation  of  the  disciplined  oscillator  is  determined.  Plots  of  Fig.  1,  2  and  3  show  samples  of  the  daily 
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normalized  frequency  deviations  of  three  GPSDOs  computed  using  the  measurement  protocol  previously 
described,  over  a  period  of  three  months.  Some  statistics  performed  on  the  data  above  is  summarized  in 


Table  1  -  Long-term  frequency  behavior  of  the  GPSDOs 


GPSDO 

A 

OCXO 

B 

OCXO 

C 

BVA 

D 

BVA 

E 

Rb 

F 

Rb 

G 

Rb 

H 

Rb 

days 

of 

measurement 

98-02-11 

98-02-23 

98-06-05 

98-06-28 

98-04-23 

98-05-08 

98-05-08 

98-05-25 

98-03-14 

98-03-27 

98-04-01 

98-04-22 

98-04-08 

98-05-04 

98-07-13 

98-08-07 

y 

relative  freq. 
deviation 

-0,9T0'13 

0,1-1  0'u 

-0,6-10" 13 

-0,2-10' 13 

-0,6- 10"13 

0,1T0"13 

3,3-10"13 

0,2  T0‘ 13 

sy 

standard  dev. 
(24  h) 

1,9  TO"12 

2,8  TO"’3 

5,  M0"13 

6,6- 10"13 

2,4 -10" 13 

4,6T0"13 

6,9- 1  O’13 

6,5-1 0"13 

Nr.  of 

samples 

12 

23 

15 

17 

13 

21 

26 

25 

Table  2  that  gives  the  standard  deviation  of  the  daily  frequency  deviations  versus  UTC(IEN),  their  upper 
an  lower  limits  and  the  number  of  samples.  The  mean  frequency  deviations  of  the  three  oscillators, 
averaged  over  the  same  observation  time,  have  not  been  reported  because  they  are  smaller  than  1  TO-13. 


Table  2  -  Statistics  on  remote  calibration  of  GPSDOs 


GPSDO 

A 

OCXO 

C 

BVA 

E 

Rb 

Sy 

Standard  dev. 

(24  h) 

0,5-10"r- 

1,1-10"13 

1 ,6- 10-13 

y  min 

-1,7-10"12 

-2.6T0"13 

-3,8-10"13 

Yinax 

1,2T0"1Z 

2,8-10"13 

4.3-10"13 

Nr.  of 

Samples 

90 

84 

79 

The  fact  that  the  standard  deviations  computed  over  24  hours  are  smaller  than  the  correspondent  ones 
listed  in  Table  1,  may  be  due  to  the  smoothing  process  performed  in  this  case  on  the  data,  originated  by 
averaging  over  the  24  hourly  data. 

From  this  long-term  analysis  of  the  GPSDOs  frequency  behavior  comes  a  confirmation  that  the 
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measurement  protocol  adopted  is  adequate  to  trace  these  devices  to  the  national  time  standard,  at  least  at 
the  level  of  parts  in  10-13,  which  is  perfectly  acceptable  in  most  calibration  centers. 


SHORT-TERM  CHARACTERIZATION  OF  GPSDOs 


In  the  frequency  calibration  field  it  is  of  the  utmost  importance  to  characterize  the  short-term  behavior  of 
the  oscillator  used  as  a  reference  in  the  calibration  process.  As  a  sample,  in  Fig.  4  to  6  have  been  reported 
the  short-term  frequency  instabilities  of  the  GPSDOs  under  test,  equipped  with  different  types  of 
oscillators,  for  averaging  times  t  of  10  s,  100  s  and  1000  s. 

The  plots  show  that  the  instantaneous  frequency  of  the  GPSDOs  can  exceed  by  orders  of  magnitude  its 
long-term  value,  and  for  x  =  1000  s  the  frequency  deviations  values  improve  significantly  over  those 
obtained  for  x=  10  s.  There  is  also  evidence  of  periodic  variations  probably  related  to  the  oscillator 
frequency  steering  process  and  to  thermal  effects. 

To  get  a  more  complete  representation  of  the  short-term  behavior  of  the  GPSDOs  considered,  some 
statistics  over  the  frequency  measurement  data  has  been  performed  and  the  results  are  reported  in 
Table  3. 

Table  3  -  Short-term  frequency  behavior  of  the  GPSDOs 


The  frequency  supplied  by  the  devices  under  test  has  been  characterized  for  averaging  times  of  1  s,  10  s, 
100  s  and  1000  s  using  a  frequency  difference  multiplier  and  an  electronic  counter.  For  each  GPSDO  has 
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been  reported  in  the  table  the  following  data: 

a)  the  Allan  deviation  (ADEV)  ay(x), 

b)  the  standard  deviation  $y  of  the  experimental  frequency  deviations  y; 

c)  the  minimum  y,™,  and  the  maximum  y^  value  of  y; 

d)  the  yp  percentile,  that  represents  the  interval  including  the  99,7%  (3ct)  of  the  experimental 

data; 

e)  the  number  of  samples. 

According  to  the  ISO  Guide  on  Uncertainty  in  Measurements  (GUM)  and  to  Document  EAL  -R2  of  the 
European  cooperation  for  Accreditation  of  Laboratories,  the  calibration  centers  must  declare  in  their 
calibration  certificates  the  expanded  uncertainty  of  the  results  obtained  as  the  standard  uncertainty 
multiplied  by  the  coverage  factor  k  =  2,  that  for  a  normal  distribution  corresponds  to  a  coverage 
probability  of  about  95%.  This  means  that  in  our  case  we  have  to  determine  the  standard  uncertainty  of 
the  experimental  data,  for  the  observation  times  commonly  used  in  calibrations,  that  satisfies  the 
previous  requirement. 

In  all  cases,  if  we  compare  the  ADEV  with  the  y^  and  yTO  values,  it  appears  that  the  first  nearly 
always  underestimates  the  real  frequency  deviations  for  every  averaging  time,  and  therefore  it  is  not  a 
reliable  representation  of  the  uncertainty  of  the  GPSDOs  to  be  used  in  the  computation  of  the  uncertainty 
budget  of  a  calibration  center.  The  same  is  also  verified  if  the  ADEV  values  are  compared  with  the  yp 
ones,  that  are  a  more  realistic  representation  of  the  behavior  of  the  oscillators  because  they  exclude  the 
possible  outliers  present  in  the  yinin  and  y^  values. 

On  the  other  hand,  also  the  standard  deviation  sy,  that  in  most  cases  copes  better  with  the  range  of 
frequency  deviations  represented  by  the  yp  values,  does  not  seem  to  satisfy  completely  to  the  criteria  of 
a  Gaussian  distribution. 

Some  tests  were  performed  on  three  sets  of  experimental  data,  relative  to  different  observation  times,  to 
check  their  probability  density  distribution;  the  corresponding  histograms  are  reported  in  Fig.  7  to  9, 
where  the  continuous  line  represents  the  Gaussian  fit  to  each  set  of  data.  There  is  evidence  for 
observation  times  of  1  s  and  10  s  that  the  fit  is  not  representative  of  the  distribution  of  the  experimental 
data,  whereas  this  works  fine  for  x  -  100  s.  But  tests  performed  on  the  same  data  sets  to  check  their 
compliance  with  a  triangular  distribution,  that  in  some  cases  seemed  to  give  a  better  interpretation  of  the 
experimental  data,  showed  that  the  Gaussian  distribution  fits  better  our  case. 

Coming  back  to  the  Allan  deviation  values,  in  the  frequency  stability  curve  shown  in  Fig.10  and 
computed  on  the  experimental  data  of  quartz  GPSDO  (A)  for  continuous  averaging  times  x  multiples  of 
x„  =  100  s,  it  can  be  observed  that  the  value  of  the  maximum  frequency  deviation 
CTy(x  =  4700  s)  >  3-CTy(x  -  100  s)  and  next  to  the  correspondent  Sy  value.  Therefore  the  uncertainty 
estimation  given  by  ADEV  versus  sy  can  be  improved  by  applying  the  same  procedure  to  all  data  sets  and 
looking  for  the  maximum  ADEV  values. 

The  behavior  observable  in  Fig.10  is  typical  of  disciplined  systems  that  show  periodic  variations  in  their 
output  frequencies  related  to  the  time  constant  implemented  in  the  disciplining  processes. 

FIELD  TRIAL  FOR  FREQUENCY 

A  first  verification  of  the  assumptions  of  above  has  been  made  during  an  interlaboratory  comparison 
organized  in  the  summer  of  1998  among  ten  calibration  centers  accredited  for  frequency  by  the  Italian 
Calibration  Service  SIT,  to  verify  the  measurement  capabilities  of  these  laboratories. 

Two  kinds  of  traveling  standards  with  different  uncertainty  levels  were  used  for  this  purpose,  a  rubidium 
and  a  high  performance  quartz  oscillator,  that  have  been  circulated  among  the  laboratories.  The  devices 
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have  been  characterized  at  the  beginning  and  at  the  end  of  the  circulation  in  the  reference  laboratory,  the 
IEN,  as  regards  to  their  frequency  deviation  and  frequency  drift.  Detailed  information  about  the  devices 
specifications,  the  measurement  procedure  and  the  uncertainty  evaluation  criteria  to  be  followed  by  the 
calibration  centers  were  also  supplied.  Each  laboratory  was  allowed  one  week  time  to  calibrate  the 
crystal  oscillator  and  two  weeks  for  the  rubidium. 

The  measurement  results  were  reported  by  the  participants  in  formal  calibration  certificates  that  have 
been  evaluated  by  IEN.  Four  out  of  the  ten  laboratories  are  equipped  with  GPSDOs  as  reference 
standards;  two  of  them  had  to  characterize  the  quartz  oscillator  and  the  others  the  rubidium  standard.  For 
both  devices,  the  frequency  deviations  reference  values  and  frequency  drift  have  been  determined  by 
IEN,  compared  with  the  calibration  data  received  and  the  compatibility  coefficient  computed. 

This  coefficient  has  been  found  compliant  (<0,5)  for  both  centers  involved  in  the  quartz  calibration,  but 
only  in  the  case  of  the  rubidium  GPSDO  the  evaluation  of  the  frequency  drift  was  reliable,  the  short-term 
frequency  deviation  of  the  quartz  GPSDO  in  fact,  as  previously  shown,  being  at  a  level  that  inhibits  the 
evaluation  of  the  quartz  daily  drift  (2,6- 10"11)  for  the  measurement  period  allowed. 

Due  to  a  failure  occurred  to  the  circulating  rubidium  that  was  replaced  afterwards  by  another  device,  the 
data  reduction  of  this  loop  has  not  yet  been  completed,  but  also  in  this  case  we  expect  that  the 
compatibility  of  the  measurement  results  is  positive  in  the  case  of  the  frequency  deviation  data,  but  some 
problems  are  foreseen  for  the  drift  evaluation. 

CONCLUSIONS 

The  studies,  the  experiments  and  the  field  verifications  performed  at  IEN  and  in  other  metrological 
laboratories  [3]  on  different  types  of  GPSDOs  to  assess  their  performances  as  reliable  means  of  standard 
frequency  and  time  dissemination  and  their  traceability  to  a  national  standard  have  demonstrated  that 
this  goal  can  be  achieved. 

To  get  a  reliable  uncertainty  evaluation  on  the  frequency  deviation  values  that  can  be  reached  by  a 
GPSDO  in  the  averaging  times  from  1  s  to  1000  s,  commonly  used  in  frequency  calibrations,  it  has  been 
found  that  the  standard  deviation  or  the  maximum  value  of  ADEV  seem  to  be  the  better  estimators  to  be 
taken  into  account  for  the  uncertainty  budget  in  calibrations. 

A  field  verification  of  the  assumptions  presented  in  this  paper,  by  means  of  a  circulation  of  a  rubidium 
oscillator  between  the  calibration  centers  equipped  with  GPSDOs,  is  still  ongoing  and  the  first  results 
confirm  the  assumptions  made. 
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Normalized  frequency  deviation  vs.  UTC(IEN) 
April  -  June  1998  (April  1  =  MJD  50904) 


MJD 

Fig.  1  -  Remote  calibration  of  a  GPSDO  (A)  with  an  OCXO 

Normalized  frequency  deviation  vs.  UTC(IEN) 
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Fig.  2  -  Remote  calibration  of  a  GPSDO  (C)  with  a  BVA 

Normalized  frequency  deviation  vs.  UTC(IEN) 
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Fig.  3  -  Remote  calibration  of  a  GPSDO  (E)  with  a  Rb 
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Normalized  frequency  deviation  vs.  ITTC(IEN) 
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Normalized  frequency  deviation  vs.  UTC(IEN) 
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Normalized  frequency  deviation  vs,  LTrC(IEN) 
GPSDO:  B  (OCXO)  -  t  =  100s 


50000  100000  150000  200000  250000 

seconds 


Fig.  4  -  GPSDO  (B),  with  an  OCXO,  for 
different  measuring  times 
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Normalized  frequency  deviation  vs.  UTC(IEN) 
GPSDO:  C  (BVA)  -  i  =  10  s 
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Fig.  5  -  GPSDO  (C),  with  a  BVA,  for 
different  measuring  times 
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Normalized  frequency  deviation  vs.  UTC(IEN)  Normalized  frequency  deviation  vs  UTC(IEN) 
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Fig.  6  -  GPSDO  (G),  with  a  Rb,  for 
different  measuring  times 
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Fig.  7  -  Frequency  output  histogram  ,f  GPSDO  (A)  for  measuring  time  of  1  s 
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Questions  and  Answers 

ROBERT  DOUGLAS  (NRC):  That  is  very  beautiful  work.  I  am  interested  in  your  feelings  about  the 
statistical  control  of  the  other  variables.  Your  calibration  certificate  is  something  which  has  evaluated 
some  things,  but  there  is  a  question  of  monitoring  the  stationarity  of  things  like  multipath,  interference, 
even  spoofing,  or  the  GPS  system  itself.  I  am  wondering  how  you  integrate  your  calibration  certification 
with  a  program  for  assurance  that  these  elements,  are  in  fact,  under  statistical  control. 

FRANCO  CORDARA  (IEN):  Well,  of  course,  what  you  are  saying  is  something  that  can  be  done  in  the 
National  Lab  on  each  device.  But  I  cannot  foresee,  in  the  case  of  secondary'  centers,  that  there  are  the 
quality  of  oscillators  to  continue  this  kind  of  observation  to  improve  what  we  get.  Of  course,  there  are 
uncertain  limitations  in  the  way  of  evaluating  them.  I  would  call  it  a  good  compromise  to  leave  these 
oscillators,  after  initial  calibrations,  in  the  National  Laboratory'  completely  free  running.  So,  it  is  a 
compromise  solution  between  the  optimum  and  the  w’orst. 

I  do  not  know  if  it  has  been  clear  -  one  thing  that  is  very'  important  to  be  aware  of  is  not  to  look  in  the 
specifications  of  this  kind  of  oscillator,  only  to  the  long-term  accuracy'  of  these  devices  when  they  work  in 
the  short  term.  In  the  short  term,  you  have  even  two-times  worse  figures  you  have  to  take  into  account; 
and  to  make  a  proper  characterization,  at  least  once  for  all. 
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